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(54) Block selection review and editing system 

(57) A system for editing the hierarchical tree struc- 
ture which is created by a block selection system to cor- 
respond to a block template of a document image, 
wherein the hierarchical tree structure includes a plural- 
ity of nodes, each of which represents a block of image 
data in the block template of a document image and con- 
tains feature data defining features of the block of image 
data. The system operates to download from memory 
the hierarchical tree structure, generate and display a 
block template of a document image corresponding to 
the hierarchical tree structure in memory, select a block 
of image data to be edited in the block template of a 
document image, edit the selected block of image data 
and update the feature data in a node corresponding to 
the selected block of image data. The system deter- 
mines whether the remaining feature data in the node 
has been affected by updated feature data, and, if so, 
the remaining feature data is altered to appropriately re- 
flect the new feature of the selected block of image data. 
The system also determines whether any of the features 
of the remaining nodes have been affected by the new 
feature of the selected block of image data, and, if so, 
feature data in the affected nodes are appropriately al- 
tered to reflect the new features of its corresponding 
block of image data. 
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Description 

The present invention relates to a system for re- 
viewing and editing results of a block selection operation 
performed on a document image. More particularly the 5 
present invention relates to a system for editing a hier- 
archical tree structure which has been created as a re- 
sult of performing a block selection operation on a doc- 
ument image. 

EP-A-0567344, EP-A-0660256 and EP-A- 10 

, corresponding respectively to U.S. 
applications Serial No. 07/873,012, "Method and Appa- 
ratus For Character Recognition", Serial No. 
08/171720, "Method and Apparatus For Selecting Text 
And/Or Non-Text Blocks In A Stored Document", and is 
Serial No. 08/338,781 "Page Analysis System", by Shin- 
Ywan Wang, are incorporated herein by reference (rep- 
resentative's refs: 2251730, 2336430 and 2395730 re- 
spectively). 

Recently developed block selection techniques, 20 
such as the techniques described in U.S. Patent Appli- 
cation Serial Nos. 07/873,012 and 08/171,720, auto- 
matically analyze document images in order to distin- 
guish between different types of image data within the 
document image. The result of such a block selection 2s 
technique is used to determine the type of subsequent 
processing to be performed on the image, such as op- 
tical character recognition (OCR), data compression, 
data routing, etc. For example, image data which is des- 
ignated as text data will be subjected to OCR processing 30 
while data which is designated as picture data would not 
be subjected to OCR processing. As a result, the differ- 
ent types of image data can be input and automatically 
processed without an operator's intervention. 

An example of how a block selection technique op- 35 
erates will be discussed below with respect to Figures 
24-26. Figure 24 shows a page of a representative doc- 
ument. The document page 2401 is arranged in a two- 
column format. The page includes a title 2402, a hori- 
zontal line 241 4, text areas 2404, 2405, and 2409 which 40 
include lines of text data, halftone picture area 2407 
which includes a graphic image which is non-text, table 
block 2410 which includes text information, framed area 
2406, halftone picture area 2411 accompanied by cap- 
tion data 241 6, and picture area 241 2 and 241 5 accom- 
panied by caption data 2417 

According to the block selection techniques de- 
scribed in U.S. Patent Application Serial Nos. 
07/873,012 and 08/171,720, the disclosures of which 
are hereby incorporated by reference, each area of doc- 50 
ument page 2401 is designated in accordance with the 
type of image data obtained therein and image data is 
then segmented based on the designation. As the block 
selection program processes the document image, a hi- 
erarchical tree structure is created as shown in Figure 55 
25. 

As shown in Figure 25. hierarchical tree structure 
25C)1 contains a plurality of nodes which represent seg- 



mented blocks of image data. Each node of the tree con- 
tains feature data which defines the features of each 
block of image data in the processed document image. 
For example, the feature data may include block loca- 
tion data, size data, attribute data (image type, such as 
text, picture, table, etc.), sub-attribute data, and child 
node and parent node pointers. In the present invention, 
child (or "descendant") nodes represent blocks which 
exist entirely within a larger block. Child nodes are de- 
picted in the hierarchical tree structure as a node 
branching from a parent node, such as node 2521 which 
branches from parent node 2510. In addition to the fea- 
ture data described above, a node which represents a 
text block may also contain feature data defining the 
block's reading orientation and reading order. 

Once hierarchical tree structure 2501 has been cre- 
ated, it is stored in memory. Upon receiving a request 
to process the image data in the document image, hier- 
archical tree structure 2501 is retrieved from memory 
and image data which has been blocked together is 
processed according to the feature data stored in its cor- 
responding node. 

In addition to using the hierarchical tree structure to 
process image data of a document image, the hierarchi- 
cal tree structure is used to generate and display a com- 
prehensible format of document image 2401 on display 

2600 as shown in Figure 26. As a result, block template 

2601 of the document image is generated and displayed 
to the user on display 2600 based on the feature data 
stored in the hierarchical tree structure shown in Figure 
25. 

The block template of the document image directly 
reflects the structure of the hierarchical tree as well as 
the feature data stored in each node of the hierarchical 
tree structure. The feature data in the node is utilized to 
generate the block outline, location, and size. In addi- 
tion, feature data in each node is used to identify the 
type of image data contained within the block and, if ap- 
propriate, reading order and reading orientation. For ex- 
ample, as shown in Figure 26. block template 2601 in- 
cludes text blocks 2602, 2604, 2605 and 2609 each of 
which correspond to nodes 2502. 2504, 2505, and 2509. 
respectively. As shown, each text block includes feature 
data which designate the block as text and also define 
the block's reading order and reading orientation. Sim- 
ilarly, horizontal line block 2614, table block 2610, frame 
block 2606. halftone picture blocks 2607 and 2611 , and 
picture block 261 2 (each of which correspond to nodes 
2514, 2510, 2506, 2507, 2511 and 2512, respectively), 
include feature data which designate the type of image 
data in the block. 

Accordingly, when a block template of a document 
image is displayed to the user the user sees a compre- 
hensible format of how the document image has been 
processed by the block selection program. As a result, 
the user can readily determine whether the block selec- 
tion program has operated correctly and whether the im- 
age data will be properly processed. 
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In the foregoing block selection techniques, difficul- 
ties have been encountered. That is, erroneous desig- 
nation of image data results : thereby creating problems 
when subjecting designated image data to the wrong 
type of image processing. For example, text data which 
should be grouped into separate blocks may be grouped 
as a single block, or table data which contains text data 
may be designated as a picture block instead of a text 
block. In the foregoing examples, the errors in grouping 
and classifying the image data will erroneously deter- 
mine the processing technique to which image data is 
subjected, thereby resulting in improper processing. 

Another difficulty encountered with the block selec- 
tion techniques described above occurs when a user de- 
sires to alter the type of processing to which designated 
image data within a document image is subjected. Con- 
ventionally, once the block selection operation has been 
performed on a document image, any editing of the re- 
sulting hierarchical tree structure is quite complex and 
time-consuming. 

There is therefore a need to edit blocks of image 
data created by an automatic block selection operation 
so as to modify the blocks themselves or the feature da- 
ta which defines the characteristics of the blocks. 

It is an object of the present invention to provide a 
system which can edit blocks of image data and the fea- 
ture data associated therewith. Preferably, editing takes 
place in the displayed block template of the document 
image and the editing program automatically updates 
feature data in the hierarchical tree structure in accord- 
ance with the editing operation. 

In one aspect of the invention, the present invention 
provides a system in which a hierarchical tree structure 
of a block template of a document image can be edited. 
According to the invention, in a block selection system 
having a hierarchical -tree structure which includes a 
plurality of nodes, each of which represents a block of 
image data in the block template of a document image 
and contains feature data defining features of the block 
of image data, there is a system for editing the hierar- 
chical tree structure. The system operates to download 
from memory the hierarchical tree structure, generate 
and display a block template of a document image cor- 
responding to the hierarchical tree structure in memory, 
select a block of image data to be edited in the block 
template of a document image, edit the selected block 
of image data and update the 'feature data in a node 
corresponding to the selected block of image data. The 
system determines whether the remaining feature data 
in the node has been affected by updated feature data, 
and, if so, the remaining feature data is altered to ap- 
propriately reflect the new feature of the selected block 
of image data. The system also determines whether any 
of the features of the remaining blocks have been af- 
fected by the new feature of the selected block of image 
data, and, if so feature data in the affected nodes are 
appropriately altered to reflect the new features of the 
corresponding blocks of image data. 



In another aspect of the invention, a block editing 
system operates to retrieve from memory a hierarchical 
tree structure, generate and display a block template of 
a document image corresponding to the hierarchical 

s tree structure, select a block of image data within the 
block template of a document image to be edited, select 
a new reading order, a reading orientation, or an at- 
tribute for the selected block of image data, and update 
the feature data defining the characteristic in a node cor- 

10 responding to the selected block of image data so as to 
conform to the selected characteristic. 

In another aspect of the invention, a block editing 
system operates to retrieve a hierarchical tree structure 
from memory, generate and display a block template of 

is a document image corresponding to the stored hierar- 
chical tree structure, display the block template of a doc- 
ument image, select at least two blocks within the doc- 
ument image to be merged together into a new block of 
image data, delete each node in the hierarchical tree 

20 structure corresponding to the previously merged 
blocks, re-display the block template of a document im- 
age having the at least two blocks of image data merged 
into the new block of image data,, and subject the new 
block of image data to a block selection operation so as 

25 to create a corresponding node in the-hierarchical tree 
structure and to store feature data in the created node. 

In another aspect of the present invention, the block 
selection system operates to retrieve a hierarchical tree 
structure from memory, generate and display a block 

30 template of a document image corresponding to the 
stored hierarchical tree structure, select a block of im- 
age data within the block template of the document im- 
age, release the image data within the selected block, 
delete a node corresponding to the selected block in the 

35 hierarchical tree structure, manually block the released 
image data into one or more blocks of image data, per- 
form a block selection operation on the one or more 
blocks so as to create a node and feature data corre- 
sponding to each of the one or more blocks of image 

-to data, and update the hierarchical tree structure. 

In another aspect of the present invention, the block 
selection system operates to retrieve a hierarchical tree 
structure from memory, select a node in the hierarchical 
tree structure to edit, edit feature data in the selected 

~*5 node, update unedited feature data in the node to con- 
form with the feature data edited in the edit step, and 
update at least one other node in the hierarchical tree 
structure in the case the edited feature data in the se- 
lected node affects feature data in the at least one other 

50 node. 

In another aspect of the present invention, the block 
selection system operates to retrieve a hierarchial tree 
structure from memory, generate and display a hierar- 
chical block template of a document image correspond- 
55 jng to the stored hierarchial tree structure, select a block 
of image data to be edited within the hierarchial block 
template of the document image, edit the selected block 
of image data, update feature data in a node corre- 
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sponding to the edited block of image data, and re-dis- 
play the edited hierarchial block template of the docu- 
ment image. 

In another aspect of the present invention, the block 
selection system operates to retrieve a hierarchical tree 5 
structure from memory, generate and display a block 
template of a document image corresponding to the 
stored hierarchical tree structure, select a block of im- 
age data within the document image to be split into at 
least two blocks of image data, designate a portion to 10 
be split from the selected block of image data, split the 
block of image data into at least two blocks of image 
data, delete a node in the hierarchical tree structure cor- 
responding to the selected block of image data, redis- 
play the block template of the document image having 
the block of image data split into at least two blocks of 
image data, and subject the at least two blocks of image 
data to block selection so as to create corresponding 
nodes in the hierarchical tree structure and to store fea- 
ture data in the created nodes. 20 

This brief summary of the invention has been pro- 
vided so that the nature of the invention may be under- 
stood quickly. A more complete understanding of the in- 
vention and its advantages may be obtained by refer- 
ence to the following detailed description in connection 25 
with the appended drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a perspective view showing the outward 
appearance of an apparatus according to the 
present invention: 

Figure 2 is a block diagram of the Figure 1 appara- 
tus; 

Figure 3, comprising Figures 3A'and 3B ; is a flow 35 
diagram describing an overview of the block editing 
system according to the present invention: 
Figure 4 is a flow diagram describing the method of 
editing the attribute of a block of image data: 
Figures 5A and 5B are representational views of a -w 
hierarchial tree structure before and after editing the 
attribute of a block of image data: 
Figure 6 is a flow diagram describing the method of 
editing the reading order of a block of image data: 
Figures 7A and 7B are representational views of a -*5 
hierarchial tree structure before and after editing the 
reading order of a block of image data: 
Figure 8 is a flow-diagram describing the method of 
editing the reading orientation of a block of image 
data: 50 
Figures 9Aand 9B are representational views of a 
hierarchial tree structure before and after editing the 
reading orientation of a block of image data: 
Figure 10 is a flow diagram describing the method 
of creating a new block of image data from within 55 
an existing block of image data in a block template 
of a document image: 

Figures 11 A and 11 Bare representational views of 



a block template of a document image and its cor- 
responding hierarchical tree structure before a new 
block of image data is created within an existing 
block- of image data; 

Figures 12A and 12B are representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure after creating 
a new block of image data within an existing block 
of image data: 

Figure 13 is a flow diagram describing the method 
of merging two blocks of image data to form a new 
block of image data in a block template of a docu- 
ment image: 

Figures 14A and 14B are representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure before two 
blocks of image data are merged to form a new 
block of image data: 

Figures 1 5A and 15B are representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure after two 
blocks of image data have been merged to form a 
new block of image data: 

Figure 16 is a flow diagram describing the method 
of releasing a block within a block template of a doc- 
ument image and creating one or more blocks from 
the image data; 

Figures 17Aand 17Bare representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure before re- 
leasing and creating one or more blocks from the 
image within the document image: 
Figures 18A and 18B are representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure after a block 
is released from the document image: 
Figures 19Aand 19Bare representational views of 
a block template of a document image and its cor- 
responding hierarchical tree structure after a block 
is created in the document image: 
Figure 20 is a flow diagram describing the method 
of low-level editing of the hierarchial tree structure: 
Figure 21 is a flow diagram describing the method 
of editing the hierarchial block template of the doc- 
ument image: 

Figures 22A and 22B are representational views of 
a hierarchial block template of a document image 
and its corresponding hierarchial tree structure prior 
to editing: 

Figures 23A and 23B are representational views of 
the hierarchial block template of the document im- 
age corresponding hierarchial tree structure after 
an editing operation: 

Figure 24 is a representational view of a document 
image: 

Figure 25 is a representational view of the hierar- 
chical tree structure which corresponds to the Fig- 
ure 24 block template of a document image: and 
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Figure 26 is a representational view of a document 
image which has undergone a block selection op- 
eration. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

Figure 1 is a view showing the outward appearance 
of a representative embodiment of the invention. Shown 
in Figure 1 is computing equipment 10\ such as a Mac- 
intosh or an IBM PC or PC-compatible computer having 
a windowing environment such as Microsoft Windows®. 
Provided with computing equipment 1 0 is display screen 
12, such as a color monitor, keyboard 13 for entering 
user commands, and pointing device 14 : such as a 
mouse, for pointing to and for manipulating objects dis- 
played on screen 12. 

Computing equipment 10 includes a mass storage 
device such as computer disk 11 for storing data files 
which include document image files, in either com- 
pressed or uncompressed format, and for storing appli- 
cation program files which can include at least one block 
selection program and a block selection editor program. 
Also stored on disk 1 1 are various hierarchial tree struc- 
ture data for each document image which has been sub- 
jected to a block selection program. Computer disk drive 
11 contains other information processing programs 
which contain stored program instructions by which 
computing equipment 1 0 manipulates the data files and 
presents data in those files to an operator via display 
screen 12. 

Document image data is input by scanner 16 which 
scans documents or other images and provides bit- 
mapped images of those documents to computing 
equipment 10. The document image data may also be 
input into computing equipment 10 from a variety of oth- 
er sources such as network interface 24 or other exter- 
nal devices via facsimile/modem interface 26. 

Printer 1 8 is provided for outputting processed doc- 
ument images. 

It should be understood that, although a program- 
mable general purpose computer arrangement is shown 
in Figure 1 , a dedicated or stand-alone computer or oth- 
er type of data processing equipment can be used to 
practice the invention. 

Figure 2 is a detailed block diagram showing the 
internal construction of computing equipment 10. As 
shown in Figure 2, computing equipment 10 includes a 
central processing unit (CPU) 20 interfaced with com- 
puter bus 21. Also interfaced with computer bus 21 is 
scanner interface 22, printer interface 23, network inter- 
face 24, fax/modem interface 26, display interface 27. 
keyboard interface 28, mouse interface 29, main mem- 
ory 30, and disk 11 . 

Main memory 30 interfaces with computer bus 21 
so as to provide random access memory storage for use 
by CPU 20 while executing stored program instructions 
such as the block selection program or the block selec- 
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tion editing program. More specifically, CPU 20 loads 
those programs from disk 11 into main memory 30 and 
executes those stored programs out of main memory 
30. 

5 In accordance with the operator instructions, stored 

application programs provide for image processing and 
manipulating of data. For example, a desktop publishing 
program such as WordPerfect for Windows© may be 
activated by an operator to create, manipulate, and view 

w documents. Likewise, a block selection operation may 
be activated to analyze the various types of image data 
in an input document image. 

Figure 3, comprising Figures 3A and 38, is a flow 
diagram describing an overview of the block editing sys- 

'5 tern. 

A preferred method for selecting a hierarchical tree 
structure and editing a block template of a document im- 
age will be described with respect to a windowing envi- 
ronment in which the user utilizes a mouse to select op- 
20 erations from menus and pull-down menus displayed in 
a window. 

Thus, in step S301, the user selects a hierarchical 
tree structure to be edited from a displayed menu. In 
step S302, the selected file which contains the hierar- 

25 chical tree structure is downloaded from disk 11 and 
stored in main memory 30. For the purpose of discus- 
sion, the hierarchical tree structure consist of text data, 
table data, halftone picture data, picture data, frame da- 
ta, and line data. 

30 In step S304, a block template of a document image 
corresponding to the selected hierarchical tree structure 
is generated from the retrieved hierarchical tree struc- 
ture and is displayed to the user. As discussed previ- 
ously each block is displayed to the user with at least 

35 the attribute feature which is defined by the feature data 
contained in its corresponding node. In step S305, the 
block selection editing program queries the user to de- 
termine whether editing is desired. If the user chooses 
not to edit the hierarchical tree structure, flow returns to 

-to step S301 , at which point the user can select a different 
hierarchical tree structure to edit. On the other hand, if 
the user elects to edit the hierarchical tree structure, 
then flow advances to step S307 in which editing oper- 
ation options are displayed to the user. 

-t5 In step S309, the user selects the editing operation 

to be performed, for example, the user could select to 
edit an attribute of a block (step S310), a reading order 
of a block (step S31 1 ), or an orientation of a block (step 
S312). The user may also choose other editing opera- 

50 tions which permit the user to redefine blocks by creat- 
ing a block within a pre-existing block or splitting the pre- 
existing block into two blocks (step S314), merging two 
blocks (step S315), releasing and creating one or more 
blocks of image data from a preexisting block (step 

55 S316). directly editing the hierarchical tree structure 
(step S317), or editing the hierarchial block template of 
the document image (step S318). 

Preferably, each of the above editing operations, 
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which will be discussed in greater detail below, are per- 
formed directly on the displayed block template of a doc- 
ument image (with the exception of step S317) and fea- 
ture data in the corresponding node/nodes are automat- 
ically updated as a result of the edit operations. 

It is noted that the following editing operations are 
merely examples of the types of editing operations 
which can be performed with the present invention. 

Attribute Editing 

Figure 4 is a detailed flow diagram showing 
processing in the event that editing of a block's attribute 
is selected (step S31 0). In the present invention, a block 
attribute defines the type of image data within the block 
created by the block selection technique. For example, 
the block attribute may define text image, picture image, 
etc. In some situations, it may be desirable to change 
the attribute set by the block selection operation, for ex- 
ample, in the case the attribute has been incorrectly des- 
ignated or in the case the user wants the block of image 
data subjected to a different type of image processing 
rather than the type designated by the attribute set by a 
block selection operation. 

Thus, in step S401 , the user utilizing mouse 1 4 : se- 
lects a particular block within the displayed block tem- 
plate of a document image to be edited. Upon selecting 
the block, flow advances to step S402 in which a menu 
of selectable attributes is displayed. The menu of se- 
lectable attributes may consist of a list of attributes 
which are arranged in lowest to highest priority or least 
to most logical selection. In step S404, the user selects 
the attribute to be assigned to the block by placing the 
cursor on the desired attribute in the menu. 

In step S405, feature data contained in the node 
representing the block is updated in accordance with the 
selected attribute. The editing of the attribute is reflected 
in the hierarchical tree structure of the block template of 
the document image. For example, as shown in Figure 
5A, there is illustrated a hierarchical tree structure of a 
block template of a document image. As shown in Figure 
5B, the user has edited the attribute in node 502 from a 
text designation to a picture designation, and the feature 
data contained in node 502 is updated to reflect the 
change. 

After the attribute has been edited, flow advances 
to step S406, at which point the block selection editing 
program determines whether the editing of the attribute 
of node 502 requires the remaining feature data within 
node 502 to be updated For example, since the unedit- 
ed version of node 502 represented text data, and the 
edited version of node 502 represents picture data, fea- 
ture data in node 502 must be updated in order to delete 
reading order and reading orientation and to create a 
sub-attribute which indicates whether the picture block 
is in joint-solid-line style. In this case, flow advances to 
step S407, so that various steps of block selection can 
be applied to the edited block. On the other hand, in step 



S406, if the edited block does not require further editing, 
flow proceeds to step S409. 

In step S409, the block selection editing program 
determines whether remaining nodes in the hierarchical 

5 tree structure require editing due to the edited attribute 
of node 502. For example, the reading order of the text 
block represented by node 504 must be edited from 3 
to 2 due to the edited attribute. In addition, the remaining 
nodes must be rearranged within the tree structure so 

10 that the nodes representing the text blocks are rear- 
ranged sequentially according to reading order Accord- 
ingly, step S410 performs various steps of block selec- 
tion in order to edit remaining nodes of the hierarchical 
tree structure. As shown in Figure 5B, the reading order 

is of node 504 has been edited from 3 to 2, and edited 
node 502 has been moved to the far right of the hierar- 
chical tree structure. However, if the hierarchical tree 
structure does not need to be edited, flow advances to 
step S411. 

20 in step S41 1 , the user is queried as to whether any 
additional editing operations should be performed. If ad- 
ditional editing is required, flow returns to step S307. If 
no further editing is desired, then flow proceeds to step 
S412 in which the edited hierarchical- tree structure is 

25 saved to computer disk 1 1 . 

Reading Order Editing 

Figure 6 is a detailed flow diagram showing 
30 processing in the event that editing of a block's reading 
order is selected (step S311). The reading order deter- 
mines the sequence in which text blocks in a document 
image are read - The determination of reading order is 
important in facilitating OCR processing of the text 
35 block. That is, in order for OCR processing to perform 
properly, the text blocks must be processed in the cor- 
rect order. Using this editing feature, the user can edit 
a reading order of a text block in the case that reading 
orders of text blocks in a block template of a document 
40 image are incorrectly assigned by a block selection 
technique 

Thus, in step S601 , the user selects and edits the 
reading order of at least one of the text blocks in the 
displayed document image. The feature data contained 

45 in the edited node representing the edited block is up- 
dated in accordance with the edited reading orders For 
example, Figure 7A and 7B show a hierarchical tree 
structure before and after editing a reading order In the 
present example, the reading order of text block 2 has 

50 been edited from 2 to 4. Consequently, as shown in Fig- 
ure 7B, the feature data in node 702 defining the reading 
order of text block 2. is updated to reflect anew assigned 
reading order of 4. 

Flow advances to step S602, in which the block se- 

55 lection editing program determines whether the reading 
orders of any of the remaining text blocks require editing 
because of the editing of node 702. If the reading order 
of any of the remaining blocks require editing, flow pro- 
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ceeds to step S604. in which the reading orders of the 
affected blocks are edited. For example, as shown in 
Figure 7B, nodes 704 and 705 are edited to reflect a 
change in their reading order due to the editing of node 
702. On the other hand, if no further editing is required, 
flow proceeds to step S605. 

In step S605, the block selection editing program 
determines whether the hierarchical tree structure 
needs to be edited in accordance with the edited reading 
orders. The positions of the nodes representing the text 
blocks of a document image in the hierarchical tree 
structure is determined by their respective reading or- 
ders. Therefore : if the reading orders have been edited, 
the structure of the hierarchical tree will require editing. 
In this case, flow proceeds to step S606, in which the 
nodes are rearranged according to the new reading or- 
ders. For example, as shown in Figure 7Aand 7B, nodes 
702, 704 : and 705 have been rearranged in accordance 
with the edited reading orders. On the other hand, if the 
hierarchical tree structure does not need to be edited, 
flow proceeds to step S607. 

In step S607 : the user is queried as to whether any 
additional editing operations are desired. If additional 
editing is required, flow returns to step S307. However, 
if no further editing is required, flow proceeds to step 
S609 in which the edited hierarchical tree structure is 
saved to computer disk 1 1 . 

Reading Orientation Editing 

Figure 8 is a detailed flowchart showing processing 
in the event that editing of a block's reading orientation 
is selected (step S312). In step S312, a reading orien- 
tation of a text block is edited. For example, if the reading 
orientation of a text block is incorrectly assigned during 
character recognition such as in the case of Japanese 
text, which is normally read in vertical columns, being 
assigned a horizontal reading orientation, the user can 
edit the reading orientation to appropriately reflect the 
reading orientation of the text. As a result, segmentation 
of line and character data can be correctly performed. 

Thus, in step S801 , the user selects the text block 
of the document image to be reoriented. In step S802, 
the reading orientation of the selected text block is 
changed and the feature data defining the reading ori- 
entation of the text block is updated. For example, Fig- 
ures 9A and 9B show the hierarchical tree structure of 
a blocked document image before and after the reading 
orientation of a block has been edited. The reading ori- 
entation of text block 3, represented by node 904, has 
been selected for editing by the user 

In step S802, the text lines and the skew of the text 
block are reconfigured in accordance with block selec- 
tion process steps 

In step S804. it is determined whether the reading 
order of the selected text block must be edited due to 
the edited reading orientation of the selected text block. 
If s6, flow proceeds to step S805, in which the reading 



order of the selected text block is edited by applying var- 
ious steps of block selection to the edited block. In ac- 
cordance with this editing, the feature data representing 
the reading order of the selected block is updated. For 

5 example, as shown by Figures 9A and 9B, the feature 
data in node 904 which defines the reading order of text 
block 3 is updated from 3 to 4. 

In step S807, the block selection editing program 
determines whether the feature data contained in the 

10 remaining nodes in the hierarchical tree structure re- 
quires updating due to the change in node 904. If so, 
the hierarchical tree structure is edited in step S809. For 
example, as shown in Figures 9A and 9B, the feature 
data representing the reading order in node 905 is up- 

is dated from 4 to 3 due to the editing of the reading ori- 
entation of text block 3. In addition, the nodes in the hi- 
erarchical tree structure are rearranged in accordance 
with the updated reading order. 

In step S81 0, the user is queried as to whether any 

20 additional editing operations are desired, if additional 
editing is requested, flow returns to step S304. On the 
other hand, if no further editing is required, flow advanc- 
es to step S811 in which the edited tree structure is 
saved to computer disk 11 . 

25 

Creating A Block From Within A Preexisting Block 

Figure 10 is a detailed flow diagram showing 
processing in the event that creation of a block from 

30 within a preexisting block is selected (step S314). That 
is, the user may create a new block from within an ex- 
isting block or split an existing block to create a new 
block if, for example: two different types of image data 
are contained within the block. By separating out a block 

35 within an existing block, the different types of image data 
can be subjected to the most appropriate type of 
processing. 

Thus, in step S1001, the user, utilizing mouse 14, 
selects a portion of a preexisting block to be separated 
40 out and blocked. As shown in Figures 1 1 A and 1 2A, por- 
tion 1207 of block 1105 has been selected. In step 
S1002, and as shown in Figures 12A and 12B, new 
block 1207 and node 1214 are created. In the case that 
the portion of the existing block is formed by-splitting the 
^5 existing block, the new portion will be represented by a 
new node in the hierarchical tree in the same manner. 

In step S1 004, the newly created block 1 207 is sub- 
jected to various process steps of block selection oper- 
ation so as to classify the new block. In this manner, 
so feature data is created and stored in node 1214. For ex- 
ample, as shown in Figure 1 2B, the feature data in node 
1214 has been updated to define a picture image. 

In step S1005, the block selection editing program 
determines whether the hierarchical tree structure 
55 needs to be edited. For example, in step SI 006, parent 
node 1111 is edited to include a pointer to child node 
1214, and the remaining nodes remain unaltered 

In step S1 007. the user is queried as to whether any 
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additional editing operations should be performed. If ad- 
ditional editing is desired, flow returns to step S304. If 
not, flow proceeds to step S1009 in which the edited hi- 
erarchical tree structure is saved to disk 11. 

Merging Two Blocks 

Figure 13 is a detailed flow diagram showing 
processing in the event that an operation to merge two 
blocks is selected (step S315). Occasionally, after a 
block selection operation has been performed on a doc- 
ument image, two blocks which more appropriately 
should have been combined exist side by side in a block 
template of the document image. By merging the two 
blocks, the image data in the resulting new block will be 
processed as one image type. 

Thus, in step S130T the user selects two blocks 
from the block template of a document image which are 
to be merged to form a new block. In step S1302, the 
two blocks are merged in the block template of a docu- 
ment image. For example, as shown in Figure 14A, 
blocks 1401 and 1402 have been selected to be 
merged, and, in Figure 15A. the two blocks are merged 
to create new block 1 501 . 

In step S1304, node 1514 is created in the hierar- 
chical tree structure representing the block created by 
merging the two selected blocks. For example, node 
1514 has been created to represent block 1 501 I n step 
S1305, the two nodes representing the merged blocks 
are deleted from the hierarchical tree structure. For ex- 
ample, nodes 1407 and 1409, representing blocks 1401 
and 1 402, have been deleted from the hierarchical tree 
structure as shown in Figure 15B. 

In step S1306, new block 1501 is subjected to var- 
ious steps of a block selection operation and feature da- 
ta defining characteristics of new block 1 501 is created 
and stored in node 1514. For example, by applying block 
selection steps, feature data contained in node 1514 is 
created to define block 1 501 as a text block with a read- 
ing order of 1 and having a horizontal reading orienta- 
tion. 

In step S1307, it is determined whether the hierar- 
chical tree structure requires further editing in view of 
the merging of blocks 1 401 and 1 402. If so, flow advanc- 
es to step S1 309, in which the affected blocks of image 
data are subjected to block selection process steps so 
as to edit their status in accordance with the new block. 
As shown in Figures 1 4B and 1 5B, the reading order of 
text block 1 404 is edited from 3 to 2. Similarly the read- 
ing order of text block 1405 represented by node 1411 
is edited from 4 to 3. 

Finally, in step S1310. the user is queried as to 
w hether any additional editing operations should be per- 
formed. If additional editing is desired, flow returns to 
step S304. On the other hand : if no further editing is re- 
quired, flow proceeds to step S1 31 1 in which the edited 
tree structure is saved to computer disk 11 . 



Releasing And Creating One Or More Blocks From An 
Existing Block 

Figure 16 is a flow diagram showing processing in 
5 the event that releasing and creating one or more blocks 
from an existing block is selected (step S316). In this 
manner, erroneously combined image data-or image 
data which the user desires to be processed differently 
can be subjected to various processing techniques. 
w Thus, in step S1601, the user selects the block to 
be released. In step S1602, the node representing the 
selected block, as well as any descendants of the node, 
is deleted from the hierarchical tree structure. For ex- 
ample, as shown in Figures 17A, block 1704 has been 
15 selected and, as shown in Figure 18A : the portion of the 
document image circumscribed by block lines is re- 
leased. Additionally, node 1711 shown in Figure 17B is 
deleted from the hierarchical tree structure in Figure 
18b. 

20 in step S1604, the user selects an unblocked por- 
tion of the block template of a document image to be 
blocked. As shown in Figure 19A, blocks 191 6 and 1917 
circumscribe the released image data. 

In step S1 605, nodes 1919 and 1 920 are created in 

25 the hierarchical tree structure which represent the newly 
blocked areas. In step S1606, blocks 1916 and 1917 
are subjected to various steps of a block selection op- 
eration in order to create and to store feature data in 
corresponding nodes 191 9 and 1920. 

30 For example, as shown in Figure 19B, node 1919 

has been created to represent block 1916 and node 
1920 has been created to represent block 1917. After 
block selection steps have been applied to blocks 1916 
and 1917, feature data is stored in nodes 191 9 and 1920 

35 to define each block's characteristics. For example, 
block 1916 has been defined as text having a reading 
order of 3 and having a horizontal reading orientation. 

In step S1607, it is determined whether the hierar- 
chical tree structure requires further editing in view of 

40 the releasing of block 1704 and the creation of blocks 
1916 and 1917. If so. in step S1609, the affected blocks 
are subjected to various steps of a block selection op- 
eration so as to update the feature data of each affected 
block. For example, node 1712 is rearranged based on 

45 its edited reading order. And new blocks 1 916 and 1917 
are arranged in accordance with their new reading or- 
der. 

In step S1610, the user is queried as to whether any 
additional editing operations should be performed. If ad- 
so ditional editing is required, flow returns to step S304. If 
not, flow proceeds to step S1611 in which the edited hi- 
erarchical tree structure is saved to computer disk 11 . 

The foregoing method of releasing and creating one 
or more new blocks can be used to release more than 
55 one block so as to create one or more new blocks from 
the unblocked image areas. 
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Direct Editing Of The Hierarchical Tree Structure 

Figure 20 is a detailed flow chart showing process- 
ing in the event that direct editing of the hierarchical tree 
structure is selected (step S317). This feature permits s 
advanced users of image processing systems to directly 
edit the hierarchical tree structure which results from 
block selection of a document image. As a result, a user 
can review and edit the hierarchical tree structure direct- 
ly, without a computer-generated interface. 10 

Thus, in step S2001, the hierarchical tree structure 
representing the selected block template of a document 
image is displayed. 

In step S2002, a node to be edited is selected and 
feature data in the node is edited by entering changes 15 
via the keyboard. In this manner, either the feature data 
contained in the nodes of the hierarchical tree structure 
or the position of the nodes in the hierarchical tree struc- 
ture can be edited. For example, the feature data con- 
tained in a node can be changed from a non-text block 20 
to a text block with reading order 4 and horizontal orien- 
tation. 

In step S2004 : it is determined whether the remain- 
ing nodes in the hierarchical tree structure require edit- 
ing. If so, either the user can edit the affected nodes di- 25 
rectly or the user can choose to allow the block editing 
program to do the editing automatically. In the case the 
user allows the block editing program to perform the ed- 
iting operation, the affected blocks are subjected to var- 
ious steps of a block selection operation in step S2005. 30 
For example, the nodes will be rearranged to sequen- 
tially order the nodes according to the reading orders of 
the blocks they represent. On the other hand, the user 
can directly edit each of the affected blocks, and step 
S2004 can be repeated to determine if any other nodes 35 
still require editing. 

In step S2006, the user is queried as to whether any 
additional editing operations should be performed. If ad- 
ditional editing is requested, flow returns to step S304. 
If not, flow proceeds to step S2007 in which the edited 
tree structure is saved to computer disk 11. 

Editing of the Hierarchial Block Template of the 
Document Image 

45 

Figure 21 is a detailed flow chart showing process- 
ing in the event that editing of the hierarchial block tem- 
plate of the document image is selected (step S31 8). In 
this feature, the advanced user of image processing 
systems can directly edit not only features of blocked 50 
image data (as discussed above) : but can also edit the 
hierarchical relationship between blocks of image data 
without having to modify the hierarchical tree. As a re- 
sult, hierarchical relationships between blocked image 
data in the document image can be easily changed and 55 
the edited relationship displayed as the hierarchical tree 
structure is modified. 

' Thus, in step S2101 . the hierarchial block template 
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of the document image is generated from the retrieved 
hierarchical tree structure and is displayed. The dis- 
played hierarchical block template of the document im- 
age differs slightly from the block template image dis- 
cussed previously. Specifically, the hierarchical block 
template includes hierarchical relationship information 
displayed at the top portion of each block in the hierar- 
chical block template. The hierarchical relationship in- 
formation allows the user to readily understand the hi- 
erarchical relationship of each block of image data with- 
in the document. For example. Figures 22A and 22B 
represent a hierarchial block template of a document im- 
age and its corresponding hierarchial tree structure. As 
shown in Figure 22A, the hierarchial relationship be- 
tween text block 2201 and document 2200 is represent- 
ed in the upper left hand corner of text block 2201 as 
"1-T-1". This information indicates that text block 2201 
is first generation child node 2221, as shown in Figure 
22B, of parent node 2220. The hierarchial relationship 
for each of the blocks of image data is generated based 
on the nodal relationship of the hierarchial tree structure 
shown in Figure 22B. Thus, for example, picture block 
2206 has a hierarchial relationship to the document 
2200 and is indicated as "1-NT-2". This information in- 
dicates that picture block 2202 is the second child pic- 
ture node of parent node 2220. 

In step S2102, a block of image data in the hierar- 
chial blocked document image is selected and edited. 
The selected editing operation can be any one of the 
editing operations discussed previously. For example, 
the "releasing and creating of a new node" edit operation 
can be selected. After the edit operation, the feature da- 
ta contained in the corresponding node representing the 
edited block is updated in accordance with the edited 
reading orders. In the case that a hierarchical relation- 
ship among blocks of image data has been changed, 
the hierarchical information displayed on the affected 
block of image data will be updated as well. 

As shown in Figure 23B, the hierarchial relationship 
of picture block 2204 and text block 2205 has been ed- 
ited. As shown in Figure 23A, a new block 2301 circum- 
scribes both picture block 2204 and text block 2205 in 
a dashed block outline. As a result, the hierarchial tree 
structure, shown in Figure 22B, is edited and updated 
as illustrated in Figure 23B. 

As shown in Figure 23B, the hierarchial tree struc- 
ture has been updated to reflect the edited relationship 
of picture block 2204 and text block 2205. In the present 
example, corresponding picture node 2225 and corre- 
sponding text node 2224 are deleted from the hierarchi- 
cal tree structure and regenerated to depend from new 
picture node 2302. In this configuration, picture node 
2302 is a child node of page node 2200 and text node 
2224 and picture node 2225 are child nodes of picture 
node 2302. Once the hierarchial tree structure is edited 
in step 2102, the hierarchial block template is redis- 
played as shown in Figure 23A. 

In step S2104, it is determined whether the remain- 
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ing nodes in the hierarchial tree structure require editing 
in view of the editing of the hierarchial relationships of 
blocks 2204 and 2205. If so, in step S21 05 : the affected 
blocks of image data, are subjected to various steps of 
block selection processing so as to update the feature 5 
data of each affected block. In the present example 
shown in Figure 23B, the only change to the hierarchical 
block template and corresponding tree is the reading or- 
der of text block 2205. The reading order changes from 
M 4" to d r. 10 

In step 21 06, the user is queried as to whether any 
additional editing operations should be performed. If ad- 
ditional editing is requested, flow returns to step S304. 
If not, flow proceeds to step S2107, in which the edited 
image is saved to computer disk 11. 75 

Claims 

1 . In a block selection system which produces a hier- 20 
archical tree structure corresponding to a block 
template of a document image, wherein the hierar- 
chical tree structure includes a plurality of nodes, 
each of which represents a block of image data in 

the block template of a document image and con- 25 
tains feature data defining features of the block of 
image data, a method for editing the hierarchical 
tree structure, the method comprising the steps of: 

downloading from memory a hierarchical tree 30 
structure: 

generating and displaying a block template of 
a document image from the hierarchical tree 
structure in memory: 

selecting a block of image data in the displayed 35 

document image to be edited: 

editing a feature of the selected block of image 

data; 

updating feature data in the node correspond- 
ing to the selected block in accordance with an -*o 
editing operation of a feature of the selected 
block; and 

determining whether non-updated feature data 
in the corresponding node should be updated 
based on the feature data edited in the editing 
step, wherein., in the case that non-updated 
feature data should be updated, the non-updat- 
ed feature data is updated so as to conform with 
the edited feature of the edited block of image 
data. 50 

2. A method according to Claim 1 , further comprising: 

a second determining step for determining 
whether a feature of at least one other block of ss 
image data is affected by the editing-of the se- 
lected block of image data and ; in the case that 
the at least one feature of the at least one other 



block of image data in the block template of a 
document image is affected by the editing of the 
selected block, feature data in a node corre- 
sponding to the at least one block of image data 
is updated. 

3. In a block selection system which creates a hierar- 
chical tree structure of a block template of a docu- 
ment image, wherein the hierarchical tree structure 
includes a plurality of nodes, each of which repre- 
sents a block of image data in the block template of 
a document image and contains feature data defin- 
ing features of the block of image data, a method 
for selectably editing at least one block of image da- 
ta, the method comprising the steps of: 

selecting a block template of a document image 
to be edited: 

selecting an editing operation to be performed 
on the block template of a document image, 
selecting at least one block of image data to be 
edited within the block template of a document 
image: 

performing the selected editing operation on 
the at least one block of image data: and 
updating feature data contained in the node of 
the hierarchical tree structure corresponding to 
the at least one edited block of image data. 

4. A method according to Claim 3, further comprising 
the step of automatically updating at least one other 
node in the hierarchical tree structure as a result of 
the edit operation to the at least one block of image 
data. 

5. In a block selection system which creates a hierar- 
chidal tree structure representative of a block tem- 
plate of a document image, wherein the hierarchical 
tree structure includes a plurality of nodes, each of 
which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block of image data, 
an apparatus for editing the hierarchical tree struc- 
ture comprising: 

a keyboard for inputting user commands: 

a memory for storing a hierarchial tree structure 

and process steps: and 

a processor for editing the hierarchical tree 
structure in accordance with the user com- 
mands and the stored process steps, 
wherein the processor executes the stored 
process steps in response to the user com- 
mands so as to (1 ) download from memory the 
hierarchical tree structure, (2) generate and 
display a block template of a document image 
corresponding to the stored hierarchical tree 
structure, (3) select at least one block of image 
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data to be edited in the block template of a doc- 
ument image, (4) edit a feature in the at least 
one block of image data in accordance with an 
input editing selection, and (5) update feature 
data contained in the node corresponding to the 
edited block of image data. 

An apparatus according to Claim 5, wherein the 
processor executes stored process steps in order 
to determine whether the remaining feature data in 
the node should be updated in accordance with the 
updated feature data so as to correspond to the ed- 
ited feature of the selected block of image data. 

In an apparatus according to Claim 5, wherein the 
processor executes stored process steps in order 
to determine whether a feature of at least one other 
block of image data is affected by the editing of the 
selected block of image data and : in the case it has 
been determined the feature of the at least one 
block of image data has been affected, feature data 
in a node corresponding to the at least one block of 
image data is updated. 

In a block selection system which creates a hierar- 
chical tree structure representative of a block tem- 
plate of a document image : wherein the hierarchical 
tree structure includes a plurality of nodes, each of 
which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block of image data, 
an apparatus for editing the hierarchical tree struc- 
ture comprising: 

input means for inputting user commands: 
memory means for storing a hierarchial tree 
structure and process steps: and 
processor means for editing the hierarchical 
tree structure in accordance with the user com- 
mands and the stored process steps, 
wherein the processor means executes the 
stored process steps in response to the user 
commands so as to (1) download from the 
memory means the hierarchical tree structure, 
(2) generate and display a block template of a 
document image corresponding to the stored 
hierarchical tree structure, (3) select at least 
one block of image data to be edited in the block 
template of a document image, (4) edit a fea- 
ture in the at least one block of image data in 
accordance with an input editing selection, and 
(5) update feature data contained in the node 
corresponding to the edited block of image da- 
ta. 

An apparatus according to Claim 8, wherein the 
processor means executes stored process steps in 
order to determine whether the remaining feature 



data in the node should be altered in accordance 
with the updated feature data so as to correspond 
to the edited feature of the selected block of image 
data. 

5 

10. In an apparatus according to Claim 8, wherein the 
processor means executes stored process steps in 
order to determine whether a feature of at least one 
other block of image data is affected by the editing 

10 means of the selected block of image data and, in 
the case it has been determined the feature of the 
at least one block of image data has been affected, 
feature data in a node corresponding to the at least 
one block of image data is updated. 

75 

11. In a block selection system which creates a hierar- 
chical tree structure corresponding to a block tem- 
plate of a document image, wherein the hierarchical 
tree structure includes a plurality of nodes, each of 

20 which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block of image data, a 
method for editing an attribute feature of the block 
of image data, the method comprising the steps of: 

25 

retrieving a hierarchical tree structure from 
memory; 

generating and displaying a block template of 
a document image corresponding to the stored 

30 hierarchical tree structure: 

selecting a block of image within the block tem- 
plate of a document image to be edited: 
selecting an attribute for the selected block of 
image data: and 

35 updating the feature data defining the attribute 

feature in a node corresponding to the selected 
block of image data so as to conform to the se- 
lected attribute feature. 

io 12. In a block selection system which creates a hierar- 
chical tree structure represented in a block template 
of a document image, wherein the hierarchical tree 
structure includes a plurality of nodes, each of 
which represents a block of image data in the block 

•*s template of a document image and contains feature 
data defining features of the block of image data, a 
method for editing a reading order feature of a block 
of text data, the method comprising the steps of: 

so retrieving a hierarchical tree structure from 

memory; 

generating and displaying a block template of 
a document image corresponding to the stored 
hierarchical tree structure: 
55 selecting a block of text data within the block 

template of a document image to be edited: 
displaying a menu containing a numerical read- 
ing order of each block of text data in the block 
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template of a document image; 
selecting a new numerical reading order for the 
selected block of text data; 
editing the feature data defining the reading or- 
der feature in a node corresponding to the se- s 
lected block of text data: and 
updating feature data in each node correspond- 
ing to blocks of image data which are affected 
by the edit of the reading order feature of the 
selected block of image data. 10 

13. In a block selection system which creates a hierar- 
chical tree structure represented in a block template 
of a document image, wherein the hierarchical tree 
structure includes a plurality of nodes, each of '5 
which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block of image data, a 
method for editing a reading orientation feature of 
a block of text data, the method comprising the 20 
steps of: 

retrieving a hierarchical tree structure from 
memory: 

generating and displaying a block template of 25 
a document image corresponding to the stored 
hierarchical tree structure: 
selecting a block of text data within the block 
template of a document image to be edited: 
displaying a menu containing at least two se- 30 
lectable reading orientation features: 
selecting one of the at least two reading orien- 
tation features: 

editing the feature data defining the reading ori- 
entation feature in a node corresponding to the 3$ 
selected block of image data; and 
updating remaining feature data in the node 
and each node corresponding to blocks of im- 
age data which are affected by the edit of the 
reading orientation of the selected block of text 40 
data. 

14. In a block selection system which creates a hierar- 
chical tree structure representative of a block tem- 
plate of a document image, wherein the hierarchical ■*$ 
t ree structure includes a plurality of nodes, each of 
which represents a block of image data within the 
block template of a document image and contains 
feature data defining features of the block of image 
data, a method of combining two blocks of image so 
data within the block template of a document image, 
the method comprising the steps of: 

retrieving a hierarchical tree structure from 
memory; 55 
generating and displaying a block template of 
a document image corresponding to the stored 
hierarchical tree structure: 



selecting at least two blocks within the docu- 
ment image to be merged together into a new 
block of image data; 

deleting each node in the hierarchical tree 
structure corresponding to the previously 
merged btocks; 

re-displaying the block template of the docu- 
ment image having the at least two blocks of 
image data merged into the new block of image 
data; and 

subjecting the new block of image data to block 
selection so as to create a corresponding node 
in the hierarchical tree structure and to store 
feature data in the created node. 

15. In a block selection system which creates a hierar- 
chical tree structure corresponding to a block tem- 
plate of a document image wherein the hierarchical 
tree structure includes a plurality of nodes each of 
which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block template of a 
document image, a method for releasing image da- 
ta from within a block and creating one or more 
blocks of image data from the released image data, 
the method comprising the steps of: 

retrieving a hierarchical tree structure from 
memory: 

generating and displaying a block template of 

a document image corresponding to the stored 

hierarchical tree structure: 

selecting a block of image data within a block 

template of a document image; 

releasing the image data within the selected 

block: 

deleting a node corresponding to the selected 
block in the hierarchical tree structure: 
manually blocking the released image data into 
one or more blocks of image data: 
performing a block selection operation on the 
one or more blocks so as to create a node and 
feature data corresponding to each of the one 
or more blocks of image data: and 
updating the hierarchical tree structure. 

16. In a block selection system which creates a hierar- 
chical tree structure corresponding to a block tem- 
plate of a document image wherein the hierarchical 
tree structure includes a plurality of nodes each of 
which represents a block of image data in the block 
template of a document image and contains feature 
data defining features of the block template of a 
document image, a method for directly editing the 
hierarchical tree structure, the method comprising 
the steps of: 

retrieving a hierarchical tree structure from 
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memory: 

selecting a node in the hierarchical tree struc- 
ture to edit; 

editing feature data in the selected node; 
updating unedited feature data in the node to 
conform with the feature data edited in the edit 
step; and 

updating at least one other node in the hierar- 
chical tree structure in the case the edited fea- 
ture data in the selected node affects feature 
data in the at least one other node. 

17. In a block selection system which creates a hierar- 
chical block template corresponding to a hierarchi- 
cal tree structure of a document image : wherein the 
hierarchical tree structure includes a plurality of 
nodes, each of which represents a block of image 
data in a document image and contains feature data 
defining features of the block of image data, a meth- 
od for generating the hierarchical block template 
and for selectably editing at least one block of image 
data in the hierarchical block template, the method 
comprising the steps of: 

retrieving from memory a hierarchical tree 
structure of a document image: 
generating and displaying a hierarchical block 
template based on the retrieved hierarchical 
tree structure, said displayed hierarchical tem- 
plate including blocks of image data and dis- 
played information for each block of image data 
which indicates at least image data type and a 
hierarchical relationship of each block within 
the hierarchical block template: 
selecting an editing operation to be performed 
on the block template of the document image: 
selecting at least one block of image data to be 
edited within the block template of the docu- 
ment image: 

performing the selected editing operation on 
the at least one block of image data: and 
updating feature data contained in the node of 
the hierarchical tree structure corresponding to 
the at least one edited block of image data. 

18. A method according to Claim 1 7, further comprising 
the step of updating at least one other node in the 
hierarchical tree structure as a result of an edit op- 
eration to the at least one block of image data. 

19. A method according to Claim 18, further comprising 
the step of redisplaying the hierarchical block tem- 
plate of the document image which has been edited 
in accordance with updating operations. 

20. In a block selection system which creates a hierar- 
chical tree structure corresponding to a block tem- 
plate of a document image, wherein the hierarchical 
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tree structure includes a plurality of nodes, each of 
which represents a block of image data within the 
block template of a document image and contains 
feature data defining features of the block of image 
data, a method for splitting a block of image data 
into one or more blocks of image data, the method 
comprising the steps of: 

retrieving a hierarchical tree structure from 
memory; 

generating and displaying a block template of 
a document image corresponding to the stored 
hierarchical tree structure; 
selecting a block of image data within the doc- 
ument image to be split into at least two blocks 
of image data; 

designating a portion to be split from the select- 
ed block of image data; 

splitting the block of image data into at least two 
blocks of image data in accordance with the 
designated portion: 

deleting a node in the hierarchical tree structure 
corresponding to the selected block of image 
data: 

redisplaying the block template of the docu- 
ment image having the block of image data split 
into at least two blocks of image data: and 
subjecting the at least two blocks of image data 
to block selection soas tocreate corresponding 
nodes in the hierarchical tree structure and to 
store feature data in the created nodes. 

21. A block selection method or apparatus comprising 
the features of any combination of the preceding 
claims. 
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